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Abstract 

Currently  mobile  C-E  War  Readiness  Spares  Kit  (WRSK)  computations  are 
not  automated.  They  are  determined  during  an  annual  meeting  attended  by 
system  users  and  suppliers.  However,  the  Air  Force  continues  to  study  several 
methods  and  models  to  automate  the  process.  This  thesis  reports  validation 
results  of  one  model.  The  model  combines  the  operating  and  non-operating 
failure  rates  to  predict  C-E  system  failures  for  a  specified  period  of  time. 
This  model  uses  operating  failures  distributed  over  operating  hours  and  non¬ 
operating  failures  distributed  over  system  power-up  attempts.  Validation 
results  showed  this  model  predicted  C-E  failures  as  good  as  current  methods 
during  periods  of  steady-state,  long  operating  hours;  however,  it  more  accur¬ 
ately  predicted  failures  during  periods  where  operating  and  non-operating  fail¬ 
ures  occurred.  Therefore,  the  model  can  be  applied  to  Air  Force  C-E  WRSK 
computations.  HQ  USAF/LEYS  should  develop  a  policy  to  ensure  C-E  system 
users  document  the  necessary  data  for  application  of  this  model. 


VALIDATION  OF  A  STOCHASTIC  MODEL  TO  DETERMINE  FAILURE 


RATES  FOR  COMMUNICATION-ELECTRONIC  SYSTEMS 


I.  Introduction 


Background 

This  thesis  attempts  to  validate  a  stochastic  model  for  determining  the 
failure  rates  of  communication-electronic  (C-E)  systems.  The  validation  will 
be  done  on  a  model  proposed  by  Captain  Thomas  M.  Skowronek,  in  his  1986 
thesis,  "Analysis  of  a  Stochastic  Model  to  Determine  Failure  Rates  for  Com¬ 
munication-Electronic  Systems."  This  effort  will  complete  his  research  and 
validate  the  model  for  accuracy.  If  the  model  proves  valid,  then  an  attempt 
will  be  made  to  apply  this  model  to  resolve  a  long  standing  Air  Force  problem 
of  computing  Communication-Electronic  (C-E)  war  readiness  spares  kit  (WRSK) 
requirements. 

A  WRSK  is  defined  as  the  spare  parts  and  system  components  required 
to  repair  and  maintain  the  mission  capability  of  mobile  C-E  systems  during 
the  early  days  of  war  (3:14-34).  Mobile  communications-electronic  WRSK  levels 
are  presently  determined  during  an  annual  meeting  attended  by  system  users 
and  system  component  suppliers.  They  analyze  past  data  and  input  personal 
experience  to  determine  future  WRSK  levels  (3:14-34).  All  computations  are 
done  by  hand  with  little  automation  (see  Appendix  A).  Without  a  true  under¬ 
standing  of  why  and  how  the  C-E  systems  fail  and  a  close  approximation  of 
their  failure  rate  distributions,  improvements  to  this  manual  method  of  deter¬ 
mining  WRSK  levels  appear  unlikely. 


Capt  Skowronek  tested  and  described  a  model  proposed  by  Capt  R.  D. 

Mabe,  Air  Force  Institute  of  Technology  (AFIT),  that  predicted  C-E  system 

failures  which  occurred  during  periods  of  both  on-time  and  off-time  as  a  result 

of  their  operational  environment  (1;  7;  10).  The  definitions  from  Skowronek's 

thesis  for  on-  and  off-time  apply. 

Operating  failures  are  straight  forward;  however,  the  non-operating 
failures  are  detected  during  equipment  power-up.  These  failures 
are  a  combination  of  three  specific  types.  First  there  are  dormant 
failures,  or  failures  occurring  when  the  equipment  is  not  operating, 
which  include  failures  occurring  while  the  unit  is  moving  from  one 
location  to  another.  Secondly,  there  are  those  failures  which  occur 
during  a  power-down  phase.  Finally,  there  are  power-up  failures 
caused  by  electrical  surges  in  the  equipment.  Any  of  these  three 
categories,  will  only  be  discovered  when  the  system  is  going  through 
a  power-up  cycle  (l:Sec  IV,  12).  Therefore,  although  non-operating 
failures  could  include  three  separate  failure  rates,  for  simplicity, 
they  are  all  combined  into  one  rate  based  on  power-up  cycles 
{13:5-61. 

Capt  Skowronek  described  Mabe’s  three-dimensional  solution  which  accom¬ 
modated  both  on-  and  off-time  failure  distributions.  He  showed  that  failures 
would  occur  on  a  plane  and  include  both  on-  and  off-time  rates.  The  plane 
is  described  by  the  formula: 

Z  =  (Za/XaXX)  +  (Zb/Yb)(Y)  (1) 

where:  Za  =  Expected  on-time  failures 

Xa  =  On-time  hours  in  current  period 

X  =  Transition  time  in  next  period 

Zb  =  Expected  off-time  failures 

Yb  =  Off-time  cycles  in  current  period  and 

Y  =  Transition  off-time  cycles  in  next  period. 

Figure  1  illustrates  the  three-dimensional  model.  This  formula  for  calculating 
'Z'  forecasts  the  expected  total  failures  in  a  future  time  period  based  on 
observations  for  time  and  cycles  in  the  current  operating  period  (13:24-30). 


The  model,  as  described  by  Skowronek.  appears  to  accurately  characterize 
the  C-E  environment.  Figure  2.  page  8.  depicts  a  three-dimensional  Poisson 
distribution,  and  pictorially  describes  what  is  occurring  in  the  dynamic  mobile 
C-E  environment.  Past  research  has  indicated  that  failures  do  occur  in  rela¬ 
tion  to  either  cycles,  time  or  a  combination  of  the  two  (1:  4;  9:  13:  14). 
This  research  is  discussed  in  greater  detail  in  Chapter  2.  The  histogram  in 
Figure  2  describes  the  probability  density  function  (pdf)  as  it  relates  to 
both  time  and  cycles  (13:69-70). 

Justification  for  Thesis 

The  specific  task  of  this  thesis  is  to  validate  the  stochastic  model  for 
determining  failures  for  communications-electronic  (C-E)  systems  described  by 
Skowronek.  If  the  model  can  be  validated  as  a  good  predictor  of  C-E  system 
failures  (i.e..  reliability),  it  could  then  provide  a  more  accurate  method  for 
determining  C-E  WRSK  requirements  (11;  12).  Skowronek  only  verified  the 
accuracy  and  logic  of  the  model.  He  lacked  empirical  data  to  validate  the 
model. 

Specific  Problem 

The  proposed  stochastic  model  has  not  been  validated  against  empirical 
data.  Before  the  Air  Force  Communications  Command  (AFCC)  or  Air  Force 
Logistics  Command  (AFLC)  can  use  the  model,  it  must  be  proven  valid.  The 
model  appears  to  accurately  portray  the  C-E  systems  operational  environment, 
therefore  it  should  increase  the  accuracy  of  predicting  failures  for  C-E  sys¬ 
tems.  By  using  actual  "on-time"  and  "off-time"  data  from  a  specified  period 
of  time,  an  accurate  prediction  of  future  failures  should  be  obtained  if  the 


model  is  valid. 


Research  Questions 

Three  important  questions  must  be  answered  to  ensure  the  validation 
results  are  not  flawed. 

1.  Does  the  concept  of  operating,  ("on-time")  and  non-operating,  ("off- 
time")  failures  apply  to  C-E  systems? 

2.  What  is  the  true  relationship  of  C-E  system  reliability  and  subsequent 
spare  parts  requirements? 

3.  Does  the  Skowronek  model  accurately  portray  these  relationships  and 
predict  failure  rates? 

Scope  and  Limitations  of  Research 

This  research  is  limited  to  focus  solely  on  the  failure  rate  distributions 
of  mobile  C-E  system  components.  Power-up  cycling  effects  are  more  pro¬ 
nounced  in  mobile  systems,  and  should  provide  a  stronger  validation. 

In  the  interest  of  a  strict  model  validation,  the  conditions  developed  by 
Skowronek  will  be  duplicated  as  closely  as  possible.  The  data  will  be  in  the 
two  categories  that  were  proposed  by  Skowronek:  operating  and  nonoperating 
failures.  The  terms  "off-time",  "non-operating",  and  "off-failures"  are  all 
interchangeable  and  will  generally  be  referred  to  as  off-time  failures.  The 
terms  "on-time",  "operating",  and  "on-failures"  are  also  interchangeable  and 


will  generally  be  referred  to  as  on-time  failures  (13:6). 


IL  Literature  Review 

Overview 

This  chapter  includes  a  discussion  of  Skowronek's  research  which  is  the 
basis  for  the  validation  process  of  this  thesis.  Other  topics  addressed  are: 
system  failures  that  are  corrected  by  reseating  components,  reliability  issues, 
and  using  current  field  maintenance  information  for  validation. 

Stochastic  Models 

Stochastic  models  emulate  processes  which  are  concerned  with  a  sequence 
of  events  ruled  by  probabilistic  laws  using  probabilistic  distributions  (8: v) . 
Examples  of  simple  stochastic  processes  are  the  outcomes  of  successive  trials 
of  flipping  a  coin,  the  results  of  a  learning  experiment,  or  successive  observa¬ 
tions  of  some  particular  characteristic  of  a  population.  Stochastic  models  may 
be  one-dimensional,  or  multidimensional;  for  example,  the  Brownian  Motion 
Process  and  the  Poisson  Process,  respectively  (8:11-21). 

Stochastic  models  are  concerned  with  the  distribution  of  random  variables 
over  a  suitable  parameter.  This  study's  parameter,  failure  density,  has  two 
components;  failures  distributed  over  power-up  cycles  and  failures  distributed 
over  operating  hours.  Skowronek's  model  uses  the  poisson  process  and  assumes 
all  failures  are  identically  distributed,  but  lie  in  independent  off-time  and 
on-time  distributions. 

Skowronek's  Model 

Skowronek  concluded  that  "to  calculate  failure  rates  based  on  operating 
time  only,  will  underestimate  the  true  failure  rate."  His  literature  review 
supported  the  concept  that  failures  do  occur  both  during  on-time  and  off- 
time.  Therefore,  a  stochastic  model  which  includes  on-time  and  off-time  fail- 


ures,  distributed  over  on-hours  and  power-up  cycles  respectively,  should  emu¬ 
late  the  C-E  systems  environment  quite  accurately.  Skowronek  supported  his 
position  with  documentation  from  a  Martin-Marietta  study  which  provided 
strong  evidence  that  failures  occur  during  both  on-time  and  off-time  (13:12- 
14;  1  :Sec  IV,  5-7,15,17;  l:Sec  VI, 5). 

Skowronek  described  a  three-dimensional  model  which  treated  both  on- 
time  and  off-time  failures.  He  concluded  that  these  combined  failures  occurred 
on  a  plane  described  by  the  formula: 

Z  =  (Za/Xa)(X)  +  (Zb,Yb)(Y)  (Refer  to  Figure  1)  (1) 

where: 

Za  =  Expected  on-time  failures 

Xa  =  On-time  hours  in  current  period 

X  =  Transition  time  in  next  period 

Zb  =  Expect -d  off-time  failures 

Yb  =  Off-time  cycles  in  current  period,  and 

Y  =  Transition  off-time  cycles  in  next  period. 

The  value  of  'Z',  is  the  expected  total  failures  in  the  next  time  period  based 
on  observations  for  time  and  cycles. 


r  a 


Figure  1.  Structural  Diagram  of  the  Failure  Plane  (13:24) 

The  histogram  In  Figure  2.  depicts  the  probability  for  values  other  than 
’Z’  given  that  the  location  of  ’Z’  on  the  plane  Is  the  expected  value  (Lambda) 
of  a  Poisson  distribution  solved  for  time  and  cycles.  Figure  2,  on  the  follow¬ 
ing  page  represents  a  probability  density  function  (pdf)  as  it  relates  to  both 
time  and  cycles  for  the  plane  (13:68-70).  This  model,  as  described  by  Skowro- 
nek,  appears  to  accurately  characterize  the  C-E  systems  environment. 
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Figure  2.  Three-Dimensional  Poisson  Distribution  (13:69) 


Skowronek  verified  his  model  using  hypothetical  data;  however,  he  was 
unable  to  a  validate  his  model  using  actual  maintenance  data.  His  model  per¬ 
formed  as  predicted,  producing  results  that  appeared  capable  of  representing 
the  C-E  systems  environment  more  accurately  than  previous  methods  of  pre¬ 
dicting  C-E  system  failure  rates.  Skowronek  also  concluded  that  the  failure 
rate  distributions  of  on-time  failures  over  on-hours  and  off-time  failures  over 
cycles  were  not  identically  and  independently  distributed  (13:70-71). 

The  lack  of  validation  using  actual  maintenance  data  led  the  author  to 
conduct  the  follow-up  process  of  validating  the  Skowronek  model.  The  author 
used  actual  maintenance  data  that  was  available  to  AFLC  through  the  mainten¬ 
ance  data  collection  (MDC)  system  (2).  This  data  was  collected  from  the  2d 
Combat  Control  Group  (2  CCG),  Patrick  AFB,  Florida.  Other  factors  which 
could  affect  the  accuracy  of  actual  maintenance  data  were  investigated. 


Component  Reseating 

An  article  written  by  Claude  F.  Veraa,  Vitro  Corporation,  published  in 
the  1985  Proceedings  Annual  Reliability  and  Maintainability  Symposium,  discus¬ 
sed  an  analysis  of  failures  on  major  computerized  systems.  Veraa  focuses  on 
the  concept  of  "reseating"  components.  In  reseating,  a  "failed"  circuit  card 
is  pulled  out  of  the  system  and  pushed  back-in  to  reestablish  the  contacts. 
The  circuit  card  is  not  replaced  if  reseating  fixes  the  "failed"  condition. 
Reseating  represented  a  very  high  percentage  of  failures  during  new  system 
testing.  Up  to  80%  of  the  failures  were  corrected  by  "reseating"  one  or  more 
components  during  the  early  phase  of  testing.  However,  this  figure  diminished 
to  zero  during  the  latter  phases  of  testing.  Veraa  suggested  numerous  potential 
hardware  and  software  causes  for  the  reseating  problems;  however,  all  too 
often  the  results  of  cycling  on/off  or  reseating  a  particular  component  cor¬ 
rected  the  failure  and  masked  the  real  underlying  cause  for  the  failure  ( 14;  150- 
151). 

Veraa  suggested  that  since  this  recycle  or  reseat  action  was  often  the 
"standard  operating  procedure,"  frequently  much  information  concerning  system 
failures  was  ignored  and  lost.  Veraa's  reseating  concept  is  very  similar  to 
the  environment  found  in  many  Air  Force  units  today.  The  association  is  that 
if  recycling  the  system  brings  it  "up"  ( i . e . .  turning  it  off  and  restarting  it;, 
often  very  little  else  is  done  as  a  corrective  action,  to  include  documenting 
the  temporary  failure.  This  becomes  of  particular  importance  when  the  result 
of  recycling  increases  the  stress  on  the  system.  This  increase  in  stress  as 
pointed  out  by  Skowronek  (13:14),  was  another  significant  factor  contributing 


to  the  failure  rate  of  the  C-E  system  (14:151). 


Dormant  and  Cyclical  Failure  Rates 

In  work  done  by  Edward  Demko,  Singer  Kearfot.t  Division,  the  effect  of 
dormant  and  cyclic  failure  rates  on  reliability  growth  was  considered.  He 
developed  a  simple  expression  to  incorporate  operating,  dormant  and  cyclic 
failure  rates  (4:92). 

Demko  concluded  that  the  failures  included  in  a  measurement  of  the  mean 
time  between  failure  (MTBF)  are  proportional  to  operation,  dormancy  and  the 
number  of  on/off  cycles.  He  computed  the  failure  rates  using  regression  tech¬ 
niques  to  observed  data.  The  inputs  Demko  used  were:  failures,  operating 
time,  dormant  time,  and  on/off  cycles.  Much  of  the  data  Demko's  work 
required  could  be  lost  as  a  result  of  the  reseating  conditions  described  by 
Veraa.  His  met'  id  is  dependent  on  the  documented  usage  profile  or  the 
imputed  failures.  But,  it  supports  the  opinion  that  cycling  of  electronic 
systems  has  a  significant  effect  upon  the  system  failure  rate  (4:96).  This 
effect  may  be  difficult  to  precisely  identify  if  data  collection  accuracy  does 
not  improve. 

Field  Data 

The  use  of  field  maintenance  data  to  assess  the  reliability  and  maintain¬ 
ability  of  electronic  systems  was  addressed  by  Eugene  Fiorentmo,  Rome  Air 
Development  Center,  April  1979.  In  his  report.  The  l.'se  of  Air  Force  Fie  id 
Maintenance  Data  for  R  &  M  Assessments  of  Ground  Electronic  Systems.'  he 
discussed  details  of  the  field  maintenance  data  collection  system,  data  compila¬ 
tion  procedures  and  analysis  of  the  data.  Fiorentino  pointed  out  that  R  &  M 
data  from  Air  Force  field  operations  provides  a  very  large  sample  for  R  ■<  M 
analysis.  He  investigated  areas  where  the  great  potential  of  this  lata  s.  or ■■  o 
may  be  weakened  due  to  poor  quality  or  inaccuracies  (5:v-vi). 


Florentine)  concluded  that  most  of  the  data  necessary  for  R  &  M  assess¬ 
ments  was  available  from  associated  MDC  systems.  His  recommendations,  while 
not  all  within  the  scope  of  field  maintenance  data  system  procedures,  were 
designed  to  improve  the  quality  and  usability  of  field  R  &  M  data  (5:  vi-vii). 

Improvements  to  the  MDC  system  since  Fiorentino's  study  have  not  elim¬ 
inated  all  difficulties  in  extracting  data  for  computing  component  failure 
rates  (2).  There  still  exists  the  problems  of  interpreting  or  recreating  the 
activities  that  occurred  in  association  with  a  particular  maintenance  action. 
For  example,  areas  requiring  interpretation  were:  determining  the  number  of 
cycles  preceding  a  failure;  was  a  failure  cyclical  (off-time)  or  operating  (on- 
time);  if  more  than  one  part  was  found  faulty,  which  caused  the  initial  failure 
and  which  was  an  associated  failure?  Shortcomings  in  the  MDC  system  will 
be  discussed  in  greater  detail  in  Chapter  3. 

Since  Skowronek  did  not  have  actual  maintenance  data  with  which  to 
validate  the  model,  he  proposed  the  use  of: 


Za  =  [pNo(pu)t] 


as  the  number  of  imputed  on-time  failures  in  [0,t|, 
where: 


P  = 


No  = 


Expected  (on-failures  in  [0,t]) 
Expected  (of f-f allures  in  [0 , t ] ) 

Expected  (of f-f allures  in  (0 , t ] ) 
Average  Number  of  power-ups 


(pu)  =  Expected  (power-up  cycles  in  [0,t]) 


(2) 


t 


time  in  the  program  period. 


IHHFiWi  ^mwrr^>  rJHVP'IV  V*  ' '  'J”V*  ^  '  V*  f  V*Vt7"*"  V"  f  V*  -V*  '  V  »V*.«fc-'"Lr  *.■*»  V* 


This  formula  is  based  on  cycles  and  time,  where  (p)  and  (N0)  are  constant 
and  (pu)t  varies  over  (O.t).  Skowronek  proved  summing  on-failures  (equal  to 
Formula  2)  over  hours  with  expected  off-time  failures  over  cycles  to  yield  a 
single  failure  "rate"  was  incorrect  (13:47).  Since  the  author  obtained  actual 
maintenance  data  in  which  gross  on-  and  off-failures  were  relatively  discern- 
able,  computing  an  imputed  on-failure  rate  was  not  necessary  for  the  valida¬ 
tion.  Therefore  'Za'  will  be  actual  operating  failures  experienced  in  the  vali¬ 
dation  experiment  and  not  computed  according  to  Skowronek's  formula. 


Overview 

This  chapter  discusses  the  data  collection  process,  criteria  for  selecting 
parts  to  report,  and  sorting  the  data.  How  this  information  was  used  to 
answer  Research  Question  #1  (Does  the  concept  of  operating,  ("on-time")  ana 
non-operating,  ("off-time")  failures  apply  to  C-E  systems?),  and  Research  Ques¬ 
tion  #2  (What  is  the  true  relationship  of  C-E  systems  reliability  and  subsequent 
spare  parts  requirements?),  are  discussed  in  detail.  The  Lotus  123  spreadsheet 
formulas  used  for  model  computations  are  found  in  Appendix  B.  This  chapter 
concludes  with  a  discussion  of  Mean  Absolute  Deviation  (MAD),  the  measure 
which  was  used  to  check  model  accuracy  and  answer  Research  Question  #3 
(Does  the  Skowronek  model  accurately  portray  these  relationships  and  predict 
failure  rate?). 

Data  Collection 

Data  collection  to  validate  the  model  was  a  two-fold  process,  using  data 
on  system  failures  collected  by  the  2nd  Combat  Control  Group  (2  CCG),  Patrick 
AFB.  Initially,  data  was  collected  using  a  form  developed  by  Skowronek  to 
ensure  cycles  per  day  and  operating  hours  were  accurateiv  recorded  see 
Appendix  C).  Also  recorded  was  any  svstem  or  subcomponent  causing  failures 
during  on-time  or  off-time.  Personnel  from  the  2  CCG  recommended  changes 
to  the  original  data  collection  form.  These  changes  provided  them  with  a 
better  understanding  of  the  desired  information  and  its  intended  use  in  the 
validation  process  (see  Appendix  D). 

Mext.  an  experiment  to  observe  data  collection  and  understand  the  jner.n- 
tional  environment  of  the  2  CCG  was  conducted.  The  experiment  included  data 


Kh 

ft 
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collection  by  the  unit  and  by  the  author.  Two  radar  systems  at  the  2  CCG, 
the  MPN-14  and  the  TPN-19,  were  selected  to  conduct  the  validation  of  the 
Skowronek  model.  These  are  generally  unreliable  systems  which  could  generate 
many  failures  in  a  relatively  short  time  span.  The  unit  collected  data  over 
a  two-month  period  by  observing  the  performance  of  each  end  item.  Since 
only  two  months  of  current  data  were  documented  on  the  forms  shown  in 
Appendices  C  and  D,  the  author  further  retrieved  historical  validation  data 
from  records  in  the  maintenance  data  collection  (MDC)  system.  Tables  1  and 
2  summarize  the  complete  validation  database  for  both  end  items. 

Table  1 

MPN-14  Data  Summary 


TYPE  OF 

FAILURE 

On 

Off 

More  Than  2  Failures 

7 

4 

3 

2  Failures  Or  Less 

49 

36 

13 

TOTAL  NSNs  FAILED 

56 

40 

16 

Table 

2 

TPN-19  Data 

Summary 

TYPE  OF 

FAILURE 

On 

Off 

More  Than  2  Failures 

2 

2 

0  1 

2  Failures  Or  Less 

6  m 

62 

2  I 

TOTAL  NSNs  FAILED 

66 

64 

Ad 

Nearly  six  months  of  failure  data  for  the  MPN-14  and  the  TPN-19  were 
extracted  and  used  as  the  initial  database  for  the  validation  process.  The 
total  NSNs  evaluated  represent  approximately  10%  of  the  possible  end  items 
that  could  fail.  It  should  be  noted  here  that  this  failure  data  was  the  same 
data  that  AFLC  would  use  to  predict  failure  rates  or  compute  WRSK  require¬ 
ments  for  these  C-E  systems. 

Research  Question  #1  was  answered  during  discussions  with  2  CCG  per¬ 
sonnel  and  by  information  found  during  the  literature  review.  After  speaking 
with  several  technicians  and  the  two  shop  chiefs,  the  author  felt  confident 
that  this  maintenance  data  would  be  usable  data.  However,  the  problem  of 
interpreting  the  data  in  regard  to  system  cycling  arose.  This  subject  is  discus¬ 
sed  further  in  Chapter  4. 

Data  Sorting 

Five  specific  elements  were  needed  to  solve  the  formula  for  'Z':  on-time 
failures,  off-time  failures,  total  number  of  cycles,  total  operating  hours  and 
total  possessed  hours.  These  data  points  were  sorted  into  periods  of  time 
called  "epochs,"  where  each  epoch  included  30  operating-days. 

Using  a  30  operating-day  epoch  created  four  epochs  to  forecast  and  eval¬ 
uate  for  the  six-month  data  base.  Four  epochs  allowed  the  forecasting  of 
expected  failures  over  equal  periods  of  operating  and  possessed  hours.  A 
30-operating-day  epoch  better  simulates  the  wartime  environment  where  a 
30-day  WRSK  would  be  deployed.  A  step-wise  forecast  method  was  used.  For 
example,  a  forecast  was  made  for  epoch  1,  and  then  the  actual  data  for  epoch 
1  was  used  to  evaluate  the  forecast  accuracy.  Further  validation  occurred 
by  adding  another  epoch  as  new  data  became  available. 
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Validating  the  Model 


This  phase  of  research  parallels  Skowronek’s  work,  but  differs  in  method. 
The  calculations  for  manipulating  the  model  were  done  on  the  Lotus  123 
spreadsheet  versus  the  MultiPlan  software  used  by  Skowronek.  The  data 
base  of  actual  "on-time1  and  "off-time"  failures,  for  the  period.  September 
1986  through  March  1987  replaces  Skowronek's  original  test  data  (see  Append¬ 
ices  E  and  F).  Finally,  the  data  was  organized  into  epochs. 

The  validation  process  required  the  forecasting  of  the  expected  number 
of  failures  for  each  epoch  using  the  database  provided.  These  forecasts  were 
then  compared  to  the  actual  number  of  failures  experienced  in  each  respective 
epoch.  The  model's  accuracy  was  based  on  how  closely  the  predicted  failures 
per  epoch  compared  to  the  actual  failures.  Once  the  initial  data  base  was 
manipulated  through  the  model,  further  validation  and  sensitivity  analysis  was 
possible  using  follow-on  data  received  from  the  test  unit.  The  formulas  used 
on  the  Lotus  123  spreadsheet  are  found  in  Appendix  B. 

Comparing  the  forecast  failures  to  actual  failures  documented  in  the 
data  base  validated  the  model's  accuracy.  The  Mean  Absolute  Deviation 
(MAD)  between  forecast  and  actual  values  was  selected  to  evaluate  the  model's 
prediction  accuracy.  The  MAD  is  an  effective  measure  for  a  short  range 
forecasting  model's  accuracv  (6:93-94). 

Forecasts 

Three  types  of  forecast  were  made  for  each  epoch.  Forecast  1  was  based 
on  all  failures  over  possessed  hours.  Forecast  2  was  based  on  failures  over 
un-hours,  and  Forecast  3  was  the  Skowronek  model  which  treated  both  on- 
failures  over  on-hours  and  off-failures  over  cycles. 


Skowronek  pointed  out  that  when  all  failures  were  distributed  over  oper¬ 
ating  hours,  the  resultant  failure  rate  was  significantly  lower  than  the  rate 
computed  by  all  failures  distributed  over  cycle-up  attempts  (13:68).  These 
two  methods  should  represent  the  lower  and  upper  limits,  respectively  of  a 
computed  failure  rate.  However,  for  this  experiment  all  failures  over  cycle- 
ups  was  not  computed,  because  it  represents  an  impractical  rate.  AFCC 
currently  uses  all  failures  over  possessed  time.  Its  most  likely  replacement 
would  be  all  failures  over  on-hours.  Therefore,  these  two  forecast  methods 
form  the  basis  for  the  validation  process.  The  desired  MAD  will  be  the  lowest 
value  for  Forecast  1  (all  failures  over  possessed  time),  2  (all  failures  over 
operating  time),  or  3  (as  computed  by  Formula  2). 

Comparing  MADs  for  the  results  should  indicate  the  significance  of  how 
on-time  and  off-time  failures  apply  to  C-E  systems  thereby  answering  Research 
Questions  #1  and  #3.  If  the  model's  accuracy  is  indeed  increased  over  current 
methods  of  determining  failures,  then  an  attempt  to  answer  Research  Question 
#2  could  be  made  by  calculating  a  30-day  WRSK  requirement  using  the  new 
failure  rates. 

There  were  no  unusual  aspects  or  steps  to  this  methodology.  The  method 
completes  validation  of  the  proposed  model  that  Skowronek  was  unable  to 
accomplish  due  to  a  lack  of  time  and  field  data.  If  proven  effective,  this 
model  will  oe  important  by  bringing  the  Air  Force  one  step  'loser  to  compu¬ 
terizing  its  system  for  computing  WRSK  requirements.  Further,  it  could  be 
used  for  improving  reliability  and  maintainability  of  C-E  systems. 

There  was  one  potential  hurdie  to  validating  the  Skowronek  model.  If 
the  C-E  failure  data  collected  for  this  validation  does  not  conform  to  the 


distributions  assumed  for  the  model,  then  additional  experimentation  will  be 
required  to  test  other  distributions  with  a  better  accuracy  potential. 

Summary 

Data  was  collected  by  2  CCG  maintenance  technicians  and  the  author. 
The  data  covered  a  six  month  period,  and  was  divided  into  30  operating-day 
epochs  for  analysis.  Data  was  used  to  establish  failure  rates,  which  were  then 
put  into  the  model  to  forecast  failures  for  each  epoch.  The  differences 
between  forecast  and  actual  values  were  expressed  as  Mean  Absolute  Deviations 


(MAD)  to  study  the  accuracy  of  the  model. 


IV.  Results  and  Analvsis 


Overview 


This  chapter  discusses  the  results  obtained  during  the  course  of  this 
research  effort.  In  this  chapter,  trends  and  relationships  are  analyzed.  The 
specific  environments  in  which  system  failures  occurred  were  defined  and  per¬ 
centages  of  the  failure  data  occurring  in  those  environments  were  determined. 
The  model  "fit"  to  each  failure  environment  was  analyzed  by  comparing  the 
Mean  Absolute  Deviations  (MADs)  for  the  three  forecasting  methods.  This 
chapter  concludes  with  plausible  reasons  for  the  trends  and  results.  Only 
MPN-14  data  will  be  presented  in  this  analysis.  Results  from  the  TP\:-19  were 


not  conclusive  enough  to  support  validation. 


Validation  Results 


Appendices  E  and  F  provide  the  spreadsheet  calculations  used  in  the 
model  validation  for  the  MPN-14  and  TPN-19  data,  respectively.  Three  fore¬ 
casts  were  computed:  Forecast  1  was  based  on  possessed  hours:  Forecast  2 
was  based  on  on-hours;  and  Forecast  3  used  the  Skowronek  model.  Each  svs- 


tem  data  base  was  fitted  to  a  set  of  32  specific  failure  environments  to  identi¬ 
fy  where  failures  occurred.  Figures  3  and  4  represent  the  decision  logic  for 
determining  failure  environments. 
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Figure  3.  MPN-14  Failure  Environment  Logic  Tree 
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Figure  4.  TPN-19  Failure  Environment  Logic  Tree 
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MPN-14  Failure  Environment 


Figure  3  shows  the  possible  frequency  of  MPN-14  failures.  The  failure 
environments  are  identified  by  number.  Terms  used  to  describe  the  failure 
environments  include:  "bunched"  if  the  failures  occurred  in  only  one  epoch 
and  "spaced"  if  in  more  than  one  epoch;  "downtime"  if  more  than  2°n  of  the 
operating  time  was  lost  due  to  failures,  "no  downtime"  if  less  than  2%:  "con¬ 
stant"  if  the  hours  varied  in  only  one  epoch,  "variable"  if  more  than  one 
epoch  varied;  and  "greater  than  2"  or  "2  or  less"  whichever  corresponded  to 
the  actual  total  number  of  failures.  Table  3  summarizes  the  characteristics 
of  each  off-time  failure  environment  in  which  failures  occurred.  The  number 


Table  3 

MPN- 14  Off-Time  Failure  Environment  Characteristics 


Environment 

Characteristics 

Frequency  /  %  j 

2 

Bunched,  downtime. 
Constant,  2  or  less 

5 

/  8.9 

4 

Bunched,  downtime, 
Constant,  Greater  than  2 

1 

/  1 . 7  i 

5 

Spaced,  no  downtime. 
Constant.  2  or  less 

1 

/  1 . 7  i 

6 

Bunched,  no  downtime. 
Constant.  2  or  less 

4 

/  -  .  J. 

10 

Bunched,  downtime. 
Variable,  2  or  less 

* 

3  .  5 

1 

11 

Spaced,  downtime. 
Variable,  Greater  than  2 

/  3.5 

1 

13 

Spaced,  no  downtime. 

1 

1 

/  1.7  : 

Variable,  2  or  less 

; 

"N 


of  National  Stock  Numbered  (NSN)  items  and  percentage  of  occurrence  are 
shown. 

Table  4  identifies  the  characteristics  of  the  on-time  failure  environments 
in  which  failures  occurred.  The  number  of  NSN  items  and  relative  percentage 
of  occurrence  are  also  shown  in  Table  4. 


Table  4 

MPN-14  On-time  Failure  Environments 


Environment 


Characteristics 


Frequency  /  % 


18 

Bunched,  downtime, 
Constant,  2  or  less 

15 

/  26.7 

20 

Bunched,  downtime, 
Constant,  greater  than 

2 

1  / 

1/7 

21 

Spaced,  no  downtime, 
Constant,  2  or  less 

1  / 

1.7 

22 

Bunched,  no  downtime, 
Constant,  2  or  less 

16 

/  28.5 

24 

Bunched,  no  downtime. 
Constant,  greater  than 

2 

2  / 

3.5 

26 

Bunched,  downtime, 
Variable,  2  or  less 

5  / 

CO 

V£ 

27 

Spaced,  downtime, 
Variable,  greater  than 

2 

1  / 

1 . 7 

There  were  32  possible  failure  environments  in  which  the  MPN-14  failures 
might  have  occurred.  Of  the  56  failure  data  points.  28. o0*  were  off-time 
failures.  Fourteen  of  the  32  potential  failure  environments  accommodated  all 


of  the  56  failure  data  points.  These  failure  environments  were  described  in 


Model  Fit  (MPN-14) 

Model  fit  was  evaluated  by  measuring  the  accuracy  of  the  forecasts  of 
all  N'SNs  within  each  failure  environment.  The  applicable  failure  environments 
(FE),  where  failures  occurred,  were  defined  in  Tables  3  and  4.  Tables  5 
and  6  show  the  specific  NSNs  of  failed  items  within  each  FE  for  off-time 
and  on-time,  respectively.  Also  shown  in  Tables  5  and  6  are  the  sum  of 
the  forecast  deviations  for  each  NSN  and  the  MAD  for  the  entire  FE. 


Table  5 
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Table  6 


MPN - 14  On-time  Failure  Environment  Model  Fit 


Deviations 

NSM 

Forecast  1 

Forecast  2 

Forecast  3  | 

F  E 

1  B 

3933000396719 

l 

.311628 

1 

327330 

1 

327330 

3910007397777 

1 

3 

1 

333906 

1 

333906 

3943003038222 

1 

488372 

1 

906472 

1 

306472 

4140010342183 

1 

.  3 

1 

331601 

1 

531601  | 

3833000196977 

1 

3 

1 

309338 

1 

909338 

6143000804383 

1 

.311628 

1 

327330 

1 

327330 

3930006432476 

1 

3 

1 

317339 

l 

917939 

5840006433820 

1 

3 

1 

317339 

1 

317339 

3903009390723 

l 

3 

1 

313739 

1 

313739 

3943002396399 

3 

.  0 

3 

001112 

3 

001112 

9900008069629 

1 

3 

1 

936033 

1 

336033 

3910001126794 

1 

3 

1 

3 

1 

328169 

9840008446217 

1 

3 

l 

340341 

1 

340341 

3960004239740 

1 

.311628 

1 

322932 

1 

322932 

3840010900139 

1 

3 

1 

333906 

1 

333306 

MAD 

400389 

406016 

406486 

F  E 

2  0 

3930008724303 

4 

3 

4 

819407 

4 

819407 

MAD 

1 

.12  3 

t 

204832 

1 

204832 

F  E 

2  1 

9919009939410 

3 

0 

3 

003023 

3 

009029 

MAD 

7  3 

791256 

791296 

F  E 

2  2 

9990006364191 

3 

.  0 

3 

001112 

3 

001112 

9840004836149 

l 

.  3 

300278 

l 

300278 

9840004839887 

1 

.  4  0  8  3  7  2 

900278 

1 

900278 

58400048361  10 

l 

.  3 

900278 

1 

300278 

9840010227934 

1 

.  3 

900278 

1 

300278 

382001  1061794 

1 

.  3 

300278 

l 

900278 

9899002177019 

l 

.  3 

302313 

1 

902913 

3960002339107 

1 

.  3 

300833 

1 

3  0  0  B  3  3 

9945006697444 

1 

.  3 

303074 

1 

303074 

3893002283226 

1 

.  3 

303902 

l 

309902 

3960008939402 

3 

.  0 

3 

008403 

3 

000403  | 

3903009399603 

3 

.  0 

2 

304202 

2 

304202  | 

3903006636336 

3 

.  0 

2 

104202 

2 

504202  j 

39430002^4 107 

l 

.  1 

I 

102793 

l 
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Expected  Outcome 

Forecast  3  was  expected  to  be  the  most  accurate  because  it  treats  all 
of  the  C-E  system  environments.  Forecast  3  should  equal  Forecast  2  in 
environments  of  ori-time  failures  only,  because  the  cyclical  term  of  Forecast 
3  goes  to  zero  when  cycle  failures  are  not  present  (compare  Formulas  4  and 
5).  Forecast  2  should  nearly  equal  Forecast  1  when  long,  steady-state  hours 
of  operation  are  experienced.  That  is  when  on-hours  approach  the  same 
amount  as  possessed  hours.  (Compare  Formulas  3  and  4). 


Forecast  1  = 


Total  Failures 


Possessed  Hours 


X 


( 


Expected  Number 
of 

Possessed  Hours 


(3) 


Forecast  2  = 


Total  Failures 


On-Hours 


X 


/Expected  Number' 
of 

On-Hours 


(4) 


rOn 


Forecast  3  = 


-Fails\/Expected  \  /cycle  Fails\  /Expected\ 


i  On- Hours/ \  On -Hours 


r 


Cycles 


Cycles 
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Analysis  of  Results 

From  the  tables,  the  following  trends  occur: 

1.  Forecast  1  was  the  most  acceptable.  This  was  not  expected.  It 
occurred  because  the  N1PN-14  experienced  nearly  steady-state  operations 
during  the  experiment.  If  a  more  dynamic  environment  existed  then  Forecast 


1  would  not  have  been  best. 


2.  Forecast  3  was  equal  or  better  than  Forecast  2.  This  was  a  desire- 
able  expected  outcome.  It  reflects  that  Forecast  3  describes  the  operating 
environment  and  cycling  quite  well  for  steady-state  and  dynamic  conditions. 

3.  Forecast  3  was  better  than  Forecast  1  where  there  was  no  mainten¬ 
ance  downtime  and  where  failure  events  were  spaced  out  over  more  than  one 
epoch.  Again,  this  reflects  the  ability  of  Forecast  3  to  capture  the  dynamic 
operating  conditions. 

4.  When  bunching  occurred,  none  of  the  forecasts  was  a  good  predictor. 
This  supports  the  theory;  the  Poisson  may  not  be  the  best  distribution  for  fail¬ 
ure  analysis.  Bunching  indicates  that  the  Negative  Binomial  may  be  better. 

TPN- 19  Results 

The  TPN- 19  results  were  not  shown  in  Chapter  4  because  the  data  set 
was  not  usable.  Only  three  percent  of  TPN-19  failures  were  off-time  failures. 
The  off-time  failure  environment  is  the  focus  of  this  research  and  therefore, 
no  realistic  analysis  could  be  performed  on  those  results.  However,  Appendix 
F  contains  the  model  computations  and  Appendix  G  contains  the  failure 
environment  characteristics.  In  Chapter  5  some  discussion  focuses  on  the 
TPN-19  results. 

Summary 

The  Skowronek  model  was  an  accurate  method  of  forecasting  C-E  system 
failure  rates.  The  Skowronek  model  reacted  as  expected  and  the  MAD  of  its 
forecasts  was  equal  or  lower  than  the  MAD  of  Forecast  2  which  was  based 
on  total  failures  over  on-hours.  The  results  described  in  this  chapter  were 
particularly  important  because  a  large  data  set  was  not  available.  Additional¬ 
ly,  the  data  set  was  sorted  according  to  normal  peacetime  operations  and  did 


not  contain  specific  information  regarding  the  number  of  cycles  that  a  com¬ 
ponent  experienced  or  whether  it  was  an  actual  on-time  failure  or  an  off- 
time  failure.  This  was  not  primary  source  data,  but  it  was  the  same  data 
used  by  AFLC.  To  fill  in  the  gaps,  certain  assumptions  were  made.  The 
operating  period  used  was  the  home  station  operating  hours  for  the  2  CCG. 
Off-time  failures  were  assumed  if  the  failure  occurred  within  30  minutes  of 
the  start  of  the  normal  operating  period.  At  other  times  an  off-time  failure 
was  readily  identified  through  the  MDC  data.  The  model  appeared  to  be 
valid  by  the  results  of  the  computations. 


V.  Conclusions  and  Recommendations 

Overview 

This  chapter  contains  conclusions  and  recommendations  reached  during 
the  course  of  this  research  effort.  Research  Question  #1  was  answered  by 
findings  from  the  experiment  conducted  to  obtain  data  on  the  MPN-14  and 
TPM- 1 9  radar  systems.  These  findings  are  discussed  in  detail.  Also  included 
in  this  chapter  is  an  analysis  of  the  Skowronek  model  computations  using 
actual  field  data,  which  answered  Research  Question  #3.  The  Skowronek  model 
was  examined  using  the  Mean  Absolute  Deviation  (MAD)  as  a  measure  of  acc¬ 
uracy  for  comparing  the  results  of  the  computations,  which  attempts  to  answer 
Research  Question  #2. 

Research  Summary 

The  Skowronek  model  was  verified  with  theoretical  data.  It  appeared 
as  a  more  accurate  measure  of  predicting  C-E  failure  rates  than  the  currently 
used  AFLC  method.  If  this  condition  held  true  using  actual  maintenance  data 
then  the  C-E  WRSK  computation  process  could  be  automated  versus  the  current 
manual  methods.  The  current  WRSK  computation  process  requires  annual  meet¬ 
ings  by  system  users  and  component  suppliers  to  review  past  data  and  evaluate 
inputs  from  past  experience. 

The  Skowronek  model  was  tested  using  actual  maintenance  data.  An  ade¬ 
quate  amount  of  primary  source  data  was  not  available,  therefore,  existing  data 
from  the  maintenance  data  collection  (MDC)  system  was  used  for  the  validation 
process.  The  data  was  sorted  into  epochs.  Failure  rates  were  computed  from 
the  total  data  set  as  described  in  Appendix  B.  Forecasts  were  made  for  each 
epoch.  Three  forecasting  methods  were  used.  Forecast  1  represented  the  cur- 


rentlv  used  method.  Forecast  2  represented  a  best  alternative  method  and 
Forecast  3  was  the  Skowronek  model.  The  absolute  value  of  deviations  from 
the  actual  failures  were  then  calculated,  and  the  resultant  MAD  was  used  to 
evaluate  the  model's  accuracy. 

The  results  were  tabulated  in  Chapter  4.  and  the  Skowronek  model  met 
the  expected  results.  These  expectations  were  that  the  Skowronek  model 
would  predict  as  accurate  or  even  more  accurately  than  Forecast  2.  Finally, 
in  all  failure  environments  the  Skowronek  model  was  nearly  as  accurate  as 
Forecast  1  when  there  was  little  downtime. 

General  Findings 

The  validation  process  was  hampered  by  the  fact  that  actual  data  was 
reported  on  the  forms  in  appendices  C  and  D,  for  a  total  of  only  two  months. 
This  meant  there  was  insufficient  data  available  to  conduct  a  validation.  How¬ 
ever,  the  author  examined  historical  maintenance  records  and  found  that  six 
months  of  standard  maintenance  data  was  also  available  to  use  in  validation. 
Difficulties  arose  in  extracting  a  valid  data  set  from  the  six-month  data  base. 

Four  conditions  occurred  which  might  have  affected  the  historical  data 
base.  First,  there  may  have  been  cycle-ups  of  the  system  that  were  not  relat¬ 
ed  to  a  failed  item.  -  This  data  was  lost  in  the  collection  process,  because 
routine  cycle -up  events  cere  nor  tracked.  <  Wang  mav  have  also  taken  ;us»c« 
during  maintenance  (i.e.  reseating);  however,  these  cycles  were  not  documented 
through  the  standard  MDC  process  either.  Second,  when  a  failure  occurred, 
documentation  could  not  reveal  whether  the  system  was  operating  normally 
or  in  a  power-up  cycle  at  the  moment  of  failure.  Third,  after  a  par,,  was 
replaced,  the  replacement  could  have  failed  irnmodiatelv,  before  the  system 


resumed  normal  operation,  or  a  very  short  time  after  the  system  returned  to 


operations.  These  could  constitute  either  a  cyclical,  (off-time)  failure  or  an 
on-time  failure.  To  clarify  which  failure  occurred,  the  author  assumed  the 
cyclical  failure  would  occur  if  a  replacement  spare  was  received  in  a  non-oper- 
ational  condition,  which  could  not  be  determined  until  a  power-up  was  attemp¬ 
ted.  On  the  other  hand,  if  the  replacement  spare  began  to  operate  normally, 
but  failed  a  short  time  after  the  system  was  operational,  the  failure  would 
be  an  on-time  failure.  Again,  MDC  documentation  did  not  reflect  the  correct 
failure.  Finally,  technicians  stated  that  they  might  cycle  the  system  up  to 
60  times  or  more  during  troubleshooting  in  an  attempt  to  find  a  faulty  sub¬ 
component.  This  loss  of  an  accurate  count  on  cycle  events  constitutes  a  very 
significant  potential  loss  of  data. 

However,  these  conditions  were  allowable  with  the  following  assumptions. 
Since  the  available  data  was  the  same  data  submitted  to  AFLC  via  the  MDC 
system,  it  would  be  best  to  use  the  MDC  data  for  the  validation.  Second,  the 
author  assumed  that  normal  home  station  conditions  occurred  throughout  the 
period  covered  by  the  data  base  (29  September  1986  through  25  March  1987). 

The  normal  hours  of  operation  for  the  MPN-14  were  from  0730  to  1500 
Monday  through  Friday,  and  for  the  TPN-19  from  0730  Monday  through  1500 
Friday.  As  a  result,  normal  operations  yielded  five  cycles  with  37.5  total  on- 
hours  per  week  for  the  MPN-14.  and  one  cycle  with  103.5  total  on-hours  per 
week  for  the  TPN-19.  If  failures  occurred,  then  on-hours  were  reduced,  ana 
cycles  might  be  increased  or  decreased.  The  data  was  sorted  as  outlined  in 
chapter  3,  page  16  using  the  above  parameters.  In  addition,  if  a  failure  occur¬ 
red  within  the  first  30  minutes  following  the  expected  power-up.  it  was  con¬ 
sidered  a  cyclical  failure. 


Data  Collection  Experiment 

The  results  from  the  data  collection  experiment  and  discussions  with 
2  CCG  personnel  answered  Research  Question  #1.  The  mobile  radar  system 
operational  environment  does  require  frequent  cycling-on  and  -off  of  the  sys¬ 
tem.  The  peacetime  environment  can  present  long  periods  of  time  in  which 
the  system  is  not  shut  down.  Additionally,  the  2  CCG  operated  under  commer¬ 
cial  power  at  their  home  station,  so  the  equipment  did  not  experience  frequent 
cycling  caused  by  powering  on  and  off  generators.  However,  during  wartime 
and  contingencies,  the  2  CCG  operates  from  portable  generators  which  create 
a  significantly  different  environment  of  power  fluctuations  and  uncontrollable 
losses  of  generator  power. 

Other  factors  that  dictate  the  potential  for  frequent  system  on  and  off 
cycling  include  the  natural  threats  of  the  wartime  environment.  These  threats 
come  from  enemy  radar-seeking  weapons  that  may  be  directed  toward  the  radar 
emitters  used  by  the  MPN-14  and  the  TPN-19  radar  systems.  If  threats  of 
this  nature  are  detected,  the  mobile  radar  system  must  be  shut  down  quickly 
to  avoid  damage  or  destruction. 

It  is  highly  likely  that  on  and  off  cycling  will  be  a  significant  part  of 
survivability  and  operational  procedures  for  the  mobile  C-E  units.  The  systems 
therefore,  will  be  affected  bv  on  and  off  cycling. 

C-E  Relationships 

Research  Question  #2  was  not  readily  answered.  The  true  relationships 
between  C-E  system  reliability  and  WRSK  requirements  can  best  be  resolved 
through  actually  computing  a  WRSK  requirement.  Comparing  WRSK  ijvels 
determined  by  this  computerized  method  and  by  the  current  method  (9)  could 


then  be  the  basis  for  answering  Research  Question  #2.  Generally,  as  the 


accuracy  of  failure  rate  predictions  increase  then  the  WRSK  requirements  will 
meet  the  system's  actual  spare  parts  requirements.  However,  this  can  on'- 
be  proven  by  calculating  a  30-dav  WRSK  requirement  using  the  inputs  from 
the  Skowronek  model.  The  author  did  not  have  time  to  accomplish  this  objec¬ 
tive  and  recommends  a  follow-on  to  this  thesis  to  accomplish  the  required 
research. 

Model  Accuracy 

The  answer  to  Research  Question  #3  is  directlv  related  to  the  acceptance 
of  the  failure  environment  concept  and  the  need  for  better  data  collection. 
The  results  indicate  that  the  model  was  a  good  predictor  for  dynamic  operating 
environments.  Further,  the  model  was  as  accurate  as  Forecast  2.  which  was 
based  on  total  failures  over  on-hours.  The  MPN-14  data  set  exhibited  28. 5°* 
off-time  failures.  It  must  be  remembered  that  this  figure  represents  only 
peacetime  operating  conditions  and  could  be  significantly  higher  during  contin¬ 
gencies  or  wartime  environments.  Although  Forecast  1  is  acceptable  for  the 
peacetime  environment,  it  appears  to  drive  requirements  levels  lower  than 
needed  during  contingencies  or  wartime  conditions.  Therefore,  the  Skowronek 
model  should  be  used  for  WRSK  computations  because  it  is  more  sensitive  to 
the  dynamics  of  the  C-E  environment. 

it  was  further  noted  tnat  as  downtime  decreased,  'hat  ,s  in-aour- 
approached  the  maximum  for  the  epoch,  the  model’s  accuracy  became  better 
in  nearlv  all  environments. 

Some  NS.\  items  will  experience  predominantly  on-time  failures  and  others 
may  experience  predominantly  off-time  failures.  A  forecast  method  that  essen¬ 
tially  uses  a  weighted  value  for  each  type  of  failure  during  'he  prediction 
process  would  be  beneficial  to  the  Air  Force.  An  immediate  benefit  would 


be  a  reduction  in  stoekage  of  NSNs  that  are  not  likeiy  to  fail  and  an  increase 
in  stoekage  of  those  NSNs  that  are  more  tikelv  to  fail  because  of  the  antici¬ 
pated  environment.  A  second  and  major  benefit  would  be  the  automation  of 
the  WRSK  computation  process  and  eliminate  the  need  for  manual  computations. 

TPN- 19  Conclusions 

As  stated  in  Chapter  4  the  results  of  the  TPN- 19  were  not  shown  because 
there  was  no  significant  off-time  failure  data.  The  author  believes  this  condi¬ 
tion  can  be  explained  by  three  factors:  use  of  commercial  power,  long  contin¬ 
uous  hours  of  operation  between  cycles,  and  data  collection  inaccuracies  dis¬ 
cussed  in  Chapter  2.  pages  9-12. 

First,  commercial  power  provides  a  constant  supply  of  electrical  power 
characterized  by  little  fluctuation  in  cycles  or  voltage,  as  compared  to  gas 
or  diesel  generators.  Personnel  from  the  2  CCG  agreed  that  commercial  power 
was  not  used  during  deployments  and  was  not  considered  available  during  con¬ 
tingencies.  The  use  of  commercial  power,  coupled  with  the  steady-state,  nor¬ 
mal  operating  hours  used  for  data  sorting  criteria,  may  have  significantly 
contributed  to  the  low  3%  of  off-failures  in  the  TPN- 19  data. 

Second,  normal  operations  for  the  TPN- 19  provided  only  6-8  cvcles  per 
30  operating-davs  This  small  number  drasticallv  reduces  the  opportunities 
for  rhe  svstem  to  experience  cvcle  fails.  Also.  rhe  maioritv  of  tne  failures 
were  two  or  less.  The  possibility  exists  that  more  cycling  took  place:  how¬ 
ever.  there  was  no  way  to  determine  that  from  the  MDC  data. 

Data  collection  difficulties  were  also  another  factor.  Presently,  codes 
do  not  exist  for  tracking  the  number  of  cycles  and  off-failures.  If  this 
information  was  readilv  retrievable  the  results  may  have  been  signifieantlv 
different  than  those  presented. 


Recommendations 


The  author  has  three  recommendations.  First,  the  Air  Force  shouid 
establish  a  policy  and  a  program  to  collect  the  failure  data  necessary  to  com¬ 
pute  failure  rates  considering  cycle-up  events.  As  Eugene  Fiorentino  stated 
in  1979,  the  Air  Force  field  data  system,  the  maintenance  data  collection 
(MDC)  system,  provides  most  of  the  data  required  to  calculate  reliability:  how¬ 
ever.  the  MDC  system  did  not  contain  all  the  required  data.  The  Air  Force 
could  benefit  greatly  if  the  MDC  system  provided  accurate  data  for  computing 
realistic  reliability  figures.  The  immediate  benefits  would  reduce  provisioning 
of  lesser  required  parts  and  increase  the  provisioning  of  more  urgently  needed 
parts.  Ultimately,  higher  mission  capable  rates  and  readiness  levels  would 
be  achieved  with  fewer  dollars  spent. 

Second.  It  is  strongly  recommended  that  a  follow  on  thesis  test  the  actual 
failure  rate  distributions  of  the  C-E  systems.  If  the  true  distributions  become 
known,  some  of  the  assumptions  to  failure  rate  calculations  can  be  discarded. 
Once  the  actual  distributions  are  known  model  adjustments  could  provide  even 
better  compensation  for  the  dynamic  C-E  system  environment. 

Finally,  computerized  WRSK  computations  using  data  from  the  Skowronek 
model  could  prove  that  the  entire  system  can  be  automated.  Automating  the 
system  for  WRSK  computation  could  save  a  sizeable  sum  of  money  for  the  Air 
Force.  Once  automated,  better  preplanning  of  mobility  and  contingency 
requirements  are  possible.  The  most  important  benefit  is  the  ability  to  reliably 
predict  usage  rates  based  on  the  expected  number  of  on-hours  and  cycles  the 
C-E  systems  would  operate. 

This  research  should  be  continued  and  expanded  to  benefit  not  only  the 
Air  Force,  but  also  the  other  services.  The  2  CCG  has  a  full  understanding 


of  what  was  needed  and  were  instrumental  in  providing  the  needed  data  to 
complete  this  research.  With  that  kind  of  support,  the  primary  source  data 
needed  could  be  obtained  and  further  study  on  this  subject  continued.  Bene¬ 
fits  from  further  study  appear  to  strongly  interface  with  recent  Reliability 
and  Maintainability  issues. 


Appendix  A:  Standard  Supply  Computation 

s  Explanation 

Determine  the  number  of  demands  per  part  over  an  operating  interval. 

Determine  the  total  number  of  operating  hours  over  which  the  above 
demands  were  generated.  This  is  subjective,  based  on: 


(Total  days  -  down  days  (weekends  and  holidays))  ’ 
(average  number  of  operating  hours  per  unit  per  day) 
(average  number  of  units  operational  each  day)  =  total 
operational  hours. 


Compute  the  meantime  between  demands  (MTBD)  for  each  part: 


MTBF= 


^total  operational 
i  hours  (step  2) 


'quantity  per  item\ 
of  the  part  (QPA) ) 


total  demands  for  the  part  (step  1) 


Compute  the  demands  per  flying  hour  (operational  hour)  for  each  part: 


DFH= 


MTBD 


Appendix  B:  Lotus  123  Spreadsheet  Formulas 


Column  Letter 

A. 

B. 

C. 

D. 

E. 

F. 

G. 

H. 

I. 

J. 

K. 

L. 


M. 


N. 


Definition 


National  Stock  Number:  Alpha  Numeric 
Epoch  Number:  Numeric 
Number  of  On-Hours  (t)  per  epoch:  Value 
Number  of  On-Fails  (Xt)  per  epoch:  Value 
Number  of  Cycles  ( c )  per  epoch:  Value 
Number  of  Off-Fails  (Yc)  per  epoch:  Value 
Number  of  Possessed  Hours  per  epoch:  Value 
Expected  On-Hours  E(t)  per  epoch:  Value 
Expected  Cycles  E(c)  per  epoch:  Value 
Possessed  Fail  Rate:  Formula 
Total  Failures 


(; 


1 


^Total  Possessed  Hours , 
Time  Only  Fail  Rate:  Formula 


( 


Total  Failures  \ 


, Total  On-Hours 
Cycle  Fail  Rate:  Formula 
Cycle  Fails 


(: 


Only  \ 
f  Cycles/ 


iTotal  Number  of 
Time  W/Cyc  Fail  Rate:  Formula 
rOn-Fails  \ 


C 


kOn-Hours> 

Actual  Number  of  Fails:  Value 
Forecast  1  (Possessed):  Formula 
(Possessed  Fail  Rale)  ’  (Expected  Possessed  Hours) 


0. 


Delta  1:  [(column  0)  -  (column  N)| 
Forecast  2  (Time  Only):  Formula 


Deita  2:  ((column  Q)  -  (column  N)  | 

Forecast  3  (Skowronek  model):  Formula 

(On-time\  /Expected  \  /Cycle\  /ExpectedX 

Fail  I*  I  On-Hoursl  +  I  Fail  )  '  [  Cycles  J 

Rate  /  \  E(h)  /  \Rate  /  \E(c)  / 

Delta  3:  ((column  Q)  -  (column  N)  | 

MAD  I :  Formula 

(Summation  of  Forecast  1  deviations)-  4 
MAD  II:  Formula 

(Summati''in  of  Forecast  2  deviations)-  4 
MAD  III:  Formula 


(Summation  of  Forecast  2  deviations)-  4 


Appendix  C:  Data  Form 


Data  Form 


DATE: 


1.  NOMENCLATURE  OF  END  ITEM: 


2.  NUMBER  OF  CYCLE  ATTEMPTS  PER  DAY:  (Cross-out  a  number  after 
each  attempt  is  made.  If  more  than  10.  add  in  numbers). 

1  2  3  4  5  6  7  8  9  10 

3.  RESULT  OF  EACH  ATTEMPT:  (Record  any  failure,  i.e..  critical  or  no:;- 
critical.  Circle  SUCCESS  or  FAILURE.  Add  more  if  required) 


1.  SUCCESS  or  FAILURE 
3.  SUCCESS  or  FAILURE 
5.  SUCCESS  or  FAILURE 
7.  SUCCESS  or  FAILURE 
9.  SUCCESS  or  FAILURE 


2.  SUCCESS  or  FAILURE 

4.  SUCCESS  or  FAILURE 
6.  SUCCESS  or  FAILURE 
8.  SUCCESS  or  FAILURE 
10.  SUCCESS  or  FAILURE 


4.  MSN  CAUSING  FAILED  ATTEMPT  (OR  NONE):  1. 


NOTE:  2.  3,  and  4  above  refer  to  non-operating  failures  only. 

FOR  OPERATING  FAILURES: 

1.  TOTAL  NUMBER  OF  OPERATING  FAILURES  FOR  TODAY: _ . 

2.  NSN  OF  PART  (SUBASSY  OR  LRU)  THAT  FAILED  (FOR  EACH  OPERA¬ 
TING  FAILURE):  _ . 

0.  TOTAL  ''ON-HOURS"  FOR  EACH  DA  i  I.E..  HOURS  OPERATED  \FTER 

SUCCESSFUL  POWER-UP  TODAY): _ . 

When  determining  NSN's  use  the  NSN  for  the  assembly  or  subassemblv  (LRU) 
containing  the  failed  component.  Examples:  Governor.  SF-6  tank.  RT-1168. 
or  item  that  would  normallv  be  in  WRSK. 


Appendix  D:  Data  Collection  Form  (Revised) 


SYSTEM: 


l.D.#: 


IN-GARRISON: 


DEPLOYED: 


DATE  TIME  POWER  APPLIED: 


DATE  TIME  POWER  REMOVED: 


WUC  NSN  OF  FAILED  ITEM 


STATUS 


JCN  COMMENTS 


USAGE: 


1  Sheet  will  be  used  for  each  power  application. 
For  each  item  failure  enter  WUC. 


Failure  Codes  for  Status  Column 


0=  Cold  start  Failure:  Use  this  code  when  the  failure  happens  within  approx¬ 
imately  30  minutes  of  applying  power  or  when  failure  can  be  attributes  to  rne 
aoplication  of  power. 


H=  Hot  Start  Failure:  Use  this  code  when  the  failure  happens  within  appr  >x- 
imately  15  minutes  of  applying  full  power  from  a  preheat  status. 


0=  Operational  Failure:  All  others. 


TO  SYSTEM  CHIEFS:  This  work  sheet  was  devised  to  collect  data  for 

research  efforts  dealing  with  system  failure  rates.  It  is  important  to  try  to 
collect  data  as  accurately  as  possible  to  make  determination  of  cycle  failures, 
operating  failures  and  number  of  cycles  during  the  period  covered  by  This 
report. 
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Appendix  G:  TPN  - 19  Failure  Environments  (FE) 


Off-Time  Failure  Environment  Characteristics 
Characterized  by 


Spaced,  No  Downtime, 
Constant,  2  or  less 

Bunched.  No  Downtime. 
Constant,  2  or  less 


Frequency  /  % 
1  1.5 


1 


1.5 


On-Time  Failure  Environment  Characteristics 

Characterized  by  Frequency  _  % 

Bunched.  Downtime,  16  24 

Constant.  2  or  less 

Bunched,  No  Downtime.  37  56 

Constant.  2  or  less 

Bunched,  No  Downtime,  2  3 

Constant.  Greater  than  2 

Spaced.  Downtime,  1  1.5 

Variable.  2  or  less 


Bunched.  Downtime. 
Variable,  2  or  less 
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